Using Q-Learning for Web Recommendations from Web Usage Data

With the rapid growth of the World Wide Web, the amount of information available online is increasing with an enormous pace. Recommender systems aim at pruning this information space and directing users toward the items that best meet their needs and interests. In this paper we propose a novel machine learning perspective toward the problem, based on reinforcement learning. We model the problem as Q-Learning, employing concepts and techniques commonly applied in the web usage mining domain. We propose that reinforcement learning paradigm provides an appropriate model for the recommendation problem, as well as a framework in which the system constantly interacts with the user and learns from her behavior. Our experimental evaluations support our claims and demonstrate how this approach can improve the quality of web recommendations.